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DETAILED ACTION 
Response to Amendment 

1 . The examiner withdraws the objection to claim 33 in the previous office action in light of 
Applicant's amendment to the claim. 

2. The examiner withdraws the 35 U.S.C. 1 12, second paragraph rejection of claims 32 and 
35 in the previous office action in light of Applicant's amendment to the claim. 

Information Disclosure Statement 

3. The examiner has considered the IDS filed August 28, 2003 and an initialed copy is 
included with this office action. 

Response to Arguments 

4. Applicant's arguments with respect to claims 1-39 have been considered but moot in light 
of the new grounds of rejection. 

Claim Rejections - 35 USC § 102 

5. The following is a quotation of the appropriate paragraphs of 35 U.S.C. 102 that form the 
basis for the rejections under this section made in this Office action: 

A person shall be entitled to a patent unless - 

(b) the invention was patented or described in a printed publication in this or a foreign country or in public use or on 
sale in this country, more than one year prior to the date of application for patent in the United States. 

6. Claims 13 and 14 are rejected under 35 U.S.C. 102(b) as being clearly anticipated by 
Shimizu et al., (hereinafter Shimizu), US 5450127. 

As per claim 13, Shimizu teaches: 

a visual sensor providing image data corresponding to sensed images from a writing 
surface or a second area spaced apart from the writing surface (the blackboard (col. 1, 11. 10-13, 
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is the writing surface required by the claim. Since the second area spaced apart from the writing 
surface is in the alternative, the limitation is met by Shimizu.) the visual sensor comprising a set 
of adjacent sensing elements being exposed collectively to successive portions of the image 
(figures 1 and 4, col 3, 11. 34-38); and 

a storage device for storing sensing element control values (Reference level (step S20), 
col. 6, 11. 34-35, inherently must be stored in some form of memory for it to be considered a 
reference level.) 

a controller coupled to the storage device and the visual sensor (fig. 3, element 10), the 
controller controlling a time duration of exposure of the sensing elements (col. 6, 11. 30-35) to the 
portion of image as a function of exposure to successive portions (fig. 6, col. 6, 11. 50-60). 
As per claim 14, Shimizu teaches: 

wherein a sensing element control value is provided for each sensing element for each 
successive portion of the image (col. 4, 1. 67-col. 5, 1. 2). Each element of the line CCD is 
ultimately stored as a pixel value so as to provide uniform overexposure protection or 
underexposure correction the control value must be used for each element of the line CCD as the 
element values are readout of the CCD. 

Claim Rejections - 35 USC § 103 
7. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 
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8. Claims 1-4 are rejected under 35 U.S.C. 103(a) as being unpatentable over Saund, US 
5528290, further in view of Tatsumi et al., (hereinafter Tatsumi), US 6476862 Bl. 
As per claim 1, Saund teaches: 

a visual sensor (camera, col. 3, 11. 12-13) providing image data corresponding to sensed 
images, the visual sensor selectively directed toward a first writing surface to sense a first image 
(col. 3, 11. 22-23); But Saund does not specifically teach that his camera can image multiple areas 
other than the board in front of the camera; however, Saund teaches that his camera has pan and 
tilt capabilities (Abstract). Tatsumi teaches that his camera is capable of complete rotation and 
that such features are well known in camera control(Figures 6-8). 

Saund goes on to teach: 

an image processor (which performs "center surround processing") coupled to the visual 
sensor to receive the image data from the visual sensor, the image processor capable of 
processing the image data as a function of direction (based upon a gradient operator, col. 4, 11. 
10-15) of the visual sensor toward the first writing surface or the second area. (col. 4, 11. 64-67, 
where the dead reckoning is based on position to pan/tilt of camera when image is captured). 

It would have been obvious to one of ordinary skill in the art to use the panning camera 
system of Tatsumi in a system like Saund's to be able to image successive whiteboard images of 
multiple wall surfaces to avoid having to wait for the board to be erased and the imaging to be 
continued. By allowing the camera to pan, the presentation with the use of the whiteboard 
imager can be continuous as the lecturer moves from one writing surface to another. 

As per claim 2, Saund teaches: 
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a storage device (memory, col. 3, 11. 14-15) for storing a first processing value and a 
second processing value (computational images, col. 3, 1. 15), wherein the image processor 
processes the image data of the first position using the first processing value (fig. 5, step 130), 
and wherein the image processor processes the image data of the second area using the second 
processing value (fig. 5, step 132). 

As per claim 3, Suand teaches a zoomed-in image, col. 3, 11. 19, and camera calibration 
settings are used when images are captured, col. 4, 11. 65-66. But Saund does not go into further 
detail, however, Tatsumi teaches: 

wherein the visual sensor includes a zoom lens (figure 1, element 1, col. 43-48), and 
wherein the first processing value relates to a first setting of the zoom lens and the second 
processing value relates to a second setting of the zoom lens (col. 7, 1. 65-col. 8, 1. 5). 

It would have been obvious to one of ordinary skill in the art to use the predetermined 
zoom settings of Tatsumi as inputs into the perspective transformations of Saund to reduce the 
need for further compensation by the transformation or to account for the additional distance that 
the subject matter being imaged is away from the camera. 

As per claim 4, Saund teaches: 

wherein the first processing value relates to an optical correction for distortion of the first 
image (fig. 2, step 102 and 106) and the second processing value relates to an optical correction 
for distortion of the second image (fig. 2, step 108). Saund teaches that all tiles are perspective 
corrected based upon a perspective distortion correction transformation that is calculated the first 
time and updated as the system is used (col. 5, 11. 54-60). 
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9. Claims 7, 18, 22, 26, 27 and 30 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Oliver, US 6330082 Bl, further in view of Tatsumi et al, (hereinafter Tatsumi), US 
6476862 Bl. 

As per claim 7, Oliver teaches: 

a visual sensor (scanner-converter, fig. 5, element 111) adapted to provide image data 
corresponding to sensed visual images of a writing surface (whiteboard, fig. 5, element 114); 

Oliver does not specifically teach the limitation: "or a second area spaced apart from the 
writing surface"; however, Tatsumi teaches a camera system capable of panning in multiple 
directions (figs. 6-8). 

an image processor (converter) coupled to the visual sensor to receive the image data 
from the visual sensor, the image processor capable of identifying information provided on the 
writing surface apart from the writing surface (col. 5, 11. 17-21). 

Note that the examiner is interpreting "identifying" as meaning that the system merely 
recreates the data written on the whiteboard as opposed to a blank whiteboard by reproducing the 
image data on a display screen or an associated printer. 

It would have been obvious to one of ordinary skill in the art to use the panning camera 
system of Tatsumi in a system like Olivers 1 to be able to image successive whiteboard images of 
multiple wall surfaces to avoid having to wait for the board to be erased and the imaging to be 
continued. By allowing the camera to pan, the presentation with the use of the whiteboard 
imager can be continuous as the lecturer moves from one writing surface to another. 

As per claim 18, it recites substantially the same limitations as claim 7 above and 
analogous remarks apply. 
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As per claim 22, Oliver teaches that his scanner can be located remotely from a scene to 
be imaged (col. 5, 11. 7-10), but Oliver does not teach imaging another part of a room. However, 
Tatsumi teaches: 

wherein the image capturing system includes a visual sensor disposed at the single 
location within the room to sense images of the at least one of the writing surface (figures 6-8) 
and the second area spaced apart from the writing surface. Each camera is located in a single 
location. 

As per claim 26, Oliver teaches: 

wherein the image capturing system includes a visual sensor adapted to scan the at least 
one of the writing surface or second area. (fig. 5, col. 5, 11. 13-20). 
As per claim 27, Tatusmi teaches: 

at least one of the writing surface or second area (via pan control figs. 6-8). 
Oliver teaches: 

locating an image capturing system at a second location in the room remote from the 
writing surface (Fig. 5, element 14); 

sensing a visual image of the writing surface with the image capturing system (fig. 5, 
element 111, col. 6, 11. 18-21); and 

identifying information provided on the writing surface with the image capturing system, 
(col. 5, 11. 17-21, identification is performed by recreating the image data on a display screen or 
an associated printer). 

It would have been obvious to one of ordinary skill in the art to use the panning camera 
system of Tatsumi in a system like Olivers' to be able to image successive whiteboard images of 
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multiple wall surfaces to avoid having to wait for the board to be erased and the imaging to be 
continued. By allowing the camera to pan, the presentation with the use of the whiteboard 
imager can be continuous as the lecturer moves from one writing surface to another. 
As per claim 30, Oliver teaches: 

wherein sensing includes sensing a plurality of visual images (line images, col. 5, 11. 16- 



10. Claims 5 and 6 are rejected under 35 U.S.C 103(a) as being unpatentable over Saund and 
Tatsumi, as applied to claim 1 above and further in view of Oliver, US 6330082. 

As per claim 5, Saund and Tatsumi teaches using a camera but provides no details with 
respect to the number of sensing elements, however, Oliver teaches: 

wherein the visual sensor comprises a sensing device having a plurality of sensing 
elements (col. 1, 11. 13-16, and linear photosensor, col. 5, 1. 11, typically linear photo sensors are 
an array of CCDs). 

It would have been obvious to one of ordinary skill in the art to use the scanner converter 
of Oliver in the system of Saund to insure that sequential portions of the tiled image has the 
proper data values before a high speed scan is initiated. 

As per claim 6, Oliver, teaches: 

wherein the sensing device comprises a linear array of sensing elements (col. 5, 11. 16- 



21). 

11. Claims 8, 9, 1 1, 12, 19, 28, 31, 32, and 37- 39 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Oliver and Tatsumi, as applied to claims 1 and 27 above and further in 
view of Hong et al., (hereinafter Hong), US 5764799. 



17) 
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As per claim 8, Oliver teaches that his system can scan an image of a whiteboard (col. 5, 
11. 7-10), but neither Oliver nor Tatsumi specifically state what type of processing is 
accomplished after collecting the image. Hong also scans an image, fig. 3, element 301, on a 
writing surface. However, Hong teaches: 

comprising a storage device (RAM, col. 4, 1. 11) for storing a reference visual image (col. 
4, 11. 17-19), and wherein the image processor is coupled to the storage device to access the 
reference visual image to identify information provided on the writing surface (col. 4, 11. 1 1-20, 
col. 10, 11. 43-50). 

It would have been obvious to one of ordinary skill in the art to use the optical character 
recognition methods of Hong with the remote scene scanner of Oliver to provide a system that 
can provide digitized data with respect to the scene scanned as in figure 5 of Oliver. In addition, 
Hong merely needs "any suitable image digitizer" (col. 3, 11. 51-52) to provide signals to the 
optical character recognition circuit. 

As per claim 9, Oliver teaches: 

wherein the visual sensor comprises a sensing device adapted to scan the writing surface 
(fig. 5, element 111). 

As per claim 11, Oliver teaches: 

wherein the image processor is adapted to identify an area requiring reimaging (col. 8, 11. 

38-43). 

As per claim 12, Oliver teaches: 

wherein the image processor controls the visual sensor to obtain a second visual image of 
at least the area (scan line portions, col 5, 11. 15-17), if reimaging is required, and wherein the 
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processor is adapted to combine the first-mentioned visual image with the second visual image 
(col. 5, 11. 19-20). 

As per claim 39, it recites substantially the same limitations as claim 12 above and 
analogous remarks apply. 

As per claim 19, Hong teaches: 

wherein the image capturing system including an image processor to identify information 
on the writing surface as a function of a reference visual image of the writing surface, (col. 4, 11. 
11-20, col. 10, 11. 43-50). 

As per claim 28, Oliver does not specifically teach that the identifying step is performed 
as a function of a reference image. However, Hong teaches: 

wherein identifying information includes identifying information as a function of a 
reference visual image of the writing surface, (col 4, 11. 1 1-20, col. 10, 11. 43-50). 

As per claim 3 1 , Hong teaches: 

wherein identifying includes identifying information as a function of a plurality of visual 
images (After Oliver collects the line images via CCD as does Hong, the image data must be 
processed for Optical Character Recognition, col. 3, 11. 50-56, a complete set of data is a function 
of the line images collected by the CCD). 

As per claim 32, Oliver does not teach optical character recognition. However, Hong 
teaches: 

wherein identifying includes identifying includes comparing a first visual image to a 
second visual image, (col. 4, 11. 37-43 or fig. 2, col. 4, 11. 46-50). 
As per claim 37, both Oliver and Hong teach: 
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wherein sensing comprises scanning the writing surface (Oliver, fig. 5, element 1 14, 
Hong, fig. 3, element 303). 

As per claim 38, Oliver teaches: 

wherein identifying information includes detecting an area of the writing surface 
requiring reimaging. (col. 8, 11 38-43). 

12. Claims 33-35 are rejected under 35 U.S.C. 103(a) as being unpatentable over Oliver, 
Tatsumi and Hong, as applied to claim 31 above and further in view of Inagaki, US 5999214. 

Inagaki teaches a videoconference system that includes a video camera for imaging 
participants, a plotting image input unit in a single system (Abstract). 

As per claim 33, neither Oliver nor Hong teaches a switch movable relative to a second 
location. However, Inagaki teaches: 

wherein sensing a visual image includes [imitating] initiating (remote switch responsive 
to voice or manual input, col. 12, 11. 4-10) sensing of a visual image with a switch movable 
relative to the second location (col. 12, 11. 1 1-25). 

It would have been obvious to one of ordinary skill in the art to use the voice direction 
detection feature of Inagaki in the system of Oliver, Tatsumi and Hong to allow for imaging of 
both a whiteboard surface (or plotting surface) and a person speaking without having to rely 
upon another person to direct the imaging process and to reliably image the person speaking (col. 
12, 11. 24-25). 

As per claim 34, neither Oliver, Tatsumi nor Hong teaches a selectively directable 
imaging system. However, Inagaki teaches: 
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wherein the image capturing system is selectively directable (col. 1, 11 25-31) to obtain a 
visual image of the writing surface (plotting device which is equivalent to a whiteboard as taught 
by Oliver (col. 5, 11. 7-10)) and a visual image of a second area (another participant, Inagaki, col. 
2, 11. 19-23 as in figs. 1A and IB) the method further comprising directing the image capture 
system toward the writing surface or the second area (participants or plotting device, (col. 3, 11. 
5-10), and wherein sensing includes sensing a visual image of the writing surface or the second 
area (col. 3, 11. 10-20). The prior art system as explained can either display a single participant or 
another input device and since as claimed the image capture system does not consist of one 
image capture device, the plurality of cameras taught by Inagaki teaches the claimed invention. 

It would have been obvious to one of ordinary skill in the art to use the features of 
Inagaki's video conference system with the writing surface imagers of Oliver, Tatsumi and Hong 
to provide the superimposing text that was written by the participants and to improve the 
ambience of the conference and to improve the search for a participant that is speaking. 

As per claim 35, Inagaki teaches: 

wherein positioning includes operating a switch to direct the image capture system 
toward the writing surface or the second area (col. 2, 11. 19-23). 

13. Claim 36 rejected under 35 U.S.C. 103(a) as being unpatentable over Oliver, Tatsumi, 
Hong, and Inagaki, as applied to claim 34 above and further in view of Saund. 

As per claim 36, neither Oliver Hong nor Inagaki teaches having a first and second 
processing value, however, Saund teaches: 

wherein sensing comprises providing image data corresponding to the visual image 
(tiles), and wherein the method further comprises storing a first processing value (transformation 
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value for a first pixel, fig. 4) and a second processing value (transformation value for a second 
pixel), and processing the image data using the first processing value and the second processing 
value as a function of direction (location of the tile with respect to the camera, col. 4, 11. 62-67) 
of the image capture system toward the writing surface or the second area, respectively (col. 3, 11. 
54-65 orcol 5, 11. 46-51). 

It would have been obvious to one of ordinary skill in the art to use the distortion 
compensation of Saund in the system of Oliver, Tatsumi, Hong, and Inagaki to provide for a 
system that will allow for distortion correction when the to compensate for any distortions due to 
off-axis viewpoints of the camera and uneven lightness levels across the board due to 
uncontrolled lighting conditions or reflections (Saund, col. 3, 11. 32-35). 
14. Claims 15 and 16 are rejected under 35 U.S. C 103(a) as being unpatentable over 
Shimizu, as applied to claim 13 above and further in view of Hong et al., (hereinafter Hong), US 
5764799. 

As per claim 15, Shimizu teaches that his system can scan an image of a blackboard (col. 
1, 11. 10-13), but Shimizu does not specifically state what type of processing is accomplished 
after collecting the image. Hong also scans an image, fig. 3, element 301, on a writing surface. 
However, Hong teaches: 

an image processor coupled to the visual sensor to receive the image data from the visual 
sensor (col. 3, 11. 50-56), the image processor capable of identifying information provided on the 
writing surface apart from the writing surface, (col. 4, 11. 11-20, 27-31, col. 10, 11. 43-50). 

It would have been obvious to one of ordinary skill in the art to use the optical character 
recognition methods of Hong with the remote scene scanner of Shimizu to provide a system that 
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can provide digitized data with respect to a scanned blackboard. In addition, Hong merely needs 
"any suitable image digitizer" (col. 3, 11. 51-52) to provide signals to the optical character 
recognition circuit. 

As per claim 16, Hong teaches: 

comprising a storage device for storing a reference visual image (col. 4, 11. 11-16), and 
wherein the image processor is coupled to the storage device to access the reference visual image 
(col. 4, 11. 23-28) to identify information provided on the writing surface. 

15. Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over Shimizu and 
Hong, as applied to claim 15 above and further in view of Saund. 

Neither Shimizu nor Hong teach correcting for optical distortion. However, Saund 
teaches: 

a storage device (memory, col. 3, 11. 14-15) for storing a processing value related to 
correction of the visual image for optical distortion (figs. 2 and 4), wherein the image processor 
is coupled to the storage device to access and use the processing value during image processing, 
(col. 3, 11. 54-65 or col. 5, 11. 46-51). 

It would have been obvious to one of ordinary skill in the art to use the perspective 
distortion compensation transformations of Saund in the imaging system of Shimizu and Hong to 
compensate for any distortions due to off-axis viewpoints of the imager and uneven lightness 
levels across the board due to uncontrolled lighting conditions or reflections (Saund, col. 3, 11. 
32-35). 

16. Claims 10, 20, 21 and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable over 
Oliver and Tatsumi, as applied to claim 7 above and further in view of Saund. 
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As per claim 10, Oliver does not teach compensating for distortion. However, Saund 
teaches: 

a storage device (memory, col. 3, 11. 14-15) for storing a processing value related to 
correction of the visual image for optical distortion (figs. 2 and 4) wherein the image processor is 
coupled to the storage device to access and use the processing value during image processing, 
(col. 3, 11. 54-65 or col. 5, 11. 46-51). 

It would have been obvious to one of ordinary skill in the art to use the perspective 
distortion compensation transformations of Saund in the imaging system of Oliver and Tatsumi 
to compensate for any distortions due to off-axis viewpoints of the camera and uneven lightness 
levels across the board due to uncontrolled lighting conditions or reflections (Saund, col. 3, 11. 
32-35). 

As per claim 20, Saund teaches: 

wherein the image capturing system includes a visual sensor disposed above the writing 
surface, (fig. 1, element 54, the writing surface is on the wall and the visual sensor is located 
above the wall on the ceiling). 

It would have been obvious to one of ordinary skill in the art to mount the remote scene 
imager of Oliver (col. 1, 11. 6-9) and Tatsumi to the ceiling to provide more efficient use of room 
space as shown in Suand (figure 1). Spectators could sit beneath and in front of the scanner 
without obstructing the scanners view of the whiteboard. It would have been particularly 
obvious to one of ordinary skill in the art that if the system can be used on a floor that it could 
also be mounted above the scene to be imaged. 

As per claim 21, Saund teaches: 
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wherein the visual sensor is mounted to a ceiling of the room. (fig. 1, element 54). 

As per claim 29, Oliver does not teach compensating for distortion. However, Saund 
teaches compensating for perspective distortion; therefore, Saund teaches: 

wherein sensing a visual image includes compensating for distortion of the visual image, 
(col. 3, 11. 30-33, col. 4, 1. 62-col. 5, 1. 8, col. 5, 11. 46-52). 

It would have been obvious to one of ordinary skill in the art to use the distortion 
correction features of Saund when there may be a slight misalignment of the line images of the 
scanning performed by Oliver. 

17. Claims 23-25 are rejected under 35 U.S.C. 103(a) as being unpatentable over Oliver, 
Tatsumi and Inagaki, as applied to claim 22 above and further in view of Saund. 

As per claim 23, although Oliver teaches that his camera can be remotely located from 
the image being scanned, Oliver only gives one example of where his scene scanner can be 
located, see fig. 5. Inagaki also does not specifically state where his imager can be located 
within a room. However, Saund teaches: 

wherein the visual sensor is mounted to a wall of the room. See fig. 1 . Saund teaches that 
the system can be located on the ceiling in fig. 1, but mounting the camera high on the wall 
serves as the functional equivalent of mounting the camera on a wall. Therefore, it would have 
been obvious to one of ordinary skill in the art to mount the imaging device on a wall because 
neither Oliver nor Inagaki limits the user on where to locate the imaging device and the Saund 
suggests that an alternative location and equally functional location of the imager could be on the 
ceiling. Also, Inagaki teaches that his system has a remote panhead that allows for panning of 
the camera (col. 1, 1. 27). 
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As per claim 24, Saund teaches: 



wherein the visual sensor is disposed above the writing surface (fig. 1). 



As per claim 25, Saund teaches: 



wherein the visual sensor is mounted to a ceiling of the room. (fig. 1). 



Conclusion 



18. Applicant's amendment necessitated the new ground(s) of rejection presented in this 
Office action. Accordingly, THIS ACTION IS MADE FINAL. See MPEP § 706.07(a). 
Applicant is reminded of the extension of time policy as set forth in 37 CFR 1 . 1 36(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within TWO 
MONTHS of the mailing date of this final action and the advisory action is not mailed until after 
the end of the THREE-MONTH shortened statutory period, then the shortened statutory period 
will expire on the date the advisory action is mailed, and any extension fee pursuant to 37 
CFR 1 .136(a) will be calculated from the mailing date of the advisory action. In no event, 
however, will the statutory period for reply expire later than SIX MONTHS from the date of this 
final action. 

19. Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Martin Miller whose telephone number is (703) 306-9134. The 
examiner can normally be reached on Monday-Friday, 9am-5pm until December 29, 2003 after 
which time the examiner's supervisor should be contacted. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Amelia Au can be reached on (703) 308-6604. The fax phone number for the 
organization where this application or proceeding is assigned is (703) 872-9306. 

Any inquiry of a general nature or relating to the status of this application or proceeding 
should be directed to the receptionist whose telephone number is (703) 305-3900. 



mem 



SUPERVISORY PATENT EXAMINER 
TECHNOLOGY CENTER 2600 




